feat(appkit): add testing kit - #530

Wire the testing kit as a published subpath and prove it against the first
of the two hand-rolled context stubs (the design gate):

- Add ./testing to both exports maps (dev + publishConfig) following the
  ./type-generator shape, add src/testing/index.ts to the tsdown entry, and
  declare vitest as an optional peerDependency. Build passes attw + publint;
  dist/testing/{index,mock-plugin-context,expect-stream,fixtures}.{js,d.ts}
  are emitted and vitest stays external to the main entry.
- Migrate dispatch-tool-call.test.ts: replace (plugin as any).context =
  { executeTool } with mockPluginContext. executeTool is now the REAL method,
  so the forwarded toolCallTimeoutMs is asserted through actual signal
  composition, the on-behalf-of (asUser) path is verified, and a new test
  proves the forwarded timeout actually aborts a slow toolkit tool end-to-end.

This is the primary win from the plan: executeTool's OBO and timeout paths
gain real assertions instead of a stub that proved nothing.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…Context

Replace the second and final hand-rolled stub — (plugin as any).context =
{ addRoute } — with the real PluginContext from mockPluginContext. The kit's
route recorder captures raw handlers, so the alias assertion (both
/invocations and /responses mount the same handler reference) holds against
the real class, where forwardAsyncErrors wrapping would otherwise break
reference identity.

Both context stubs the plan identified are now migrated.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

- Add docs/docs/development/testing.md covering mockPluginContext(),
  expectStream(), and the fixture helpers, with a full end-to-end example.
  Cross-links to local-development, custom-plugins, and execution-context.
- Add template/server/example.test.ts: a self-contained, plugin-agnostic
  example that scaffolded apps ship with — it defines a tiny custom plugin
  and exercises both mockPluginContext (route recording) and expectStream
  (ordered event assertions), running with no workspace or network.

Ships the kit to users, satisfying the plan's acceptance criteria that a
docs page exists and the template carries at least one example test.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Validation by scaffolding a real app with `databricks apps init` surfaced
that the examples called the `analytics()`/`toPlugin()` factory and then
treated the result as a plugin instance — but a factory returns a
{ plugin, config, name } descriptor for createApp to construct, so
`.attachContext`/handler methods are absent.

Rewrite both the template example test and the docs "Full example" to
instantiate the plugin class directly (`new GreeterPlugin({})`), matching how
the migrated agents suites use the kit. The scaffolded app's `npm test` and
`tsc` both pass against the published `@databricks/appkit/testing` subpath
with no workspace or network.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…pe error

Drop `undefined` from the static FakeToolValue union. `resolve()` treats an
undefined map entry as "unregistered tool" and throws, so allowing undefined
as a declared response made `{ query: undefined }` a confusing runtime error
instead of a compile error. A function returning undefined still works for the
rare "returns nothing" case. Add a test pinning that a null response is
returned as a value, not misread as a missing tool.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…ting kit

The plan's step 5 was to MOVE the fixtures into the package, not copy them.
The shipped kit (src/testing/fixtures.ts) duplicated all 15 exports of
tools/test-helpers.ts, which would drift over time. Collapse the original
into a thin re-export of @databricks/appkit/testing so src/testing is the
single source of truth while the 18 existing @tools/test-helpers importers
keep working unchanged.

The re-exported mockServiceContext is now synchronous; every call site either
awaits it (no-op on a non-promise) or reads it through
Awaited<ReturnType<...>>, so all suites pass unchanged (full appkit suite:
3117 passed, 1 pre-existing skip).

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…ing docs

Code review follow-ups:

- expectStream's parseSSEBody split frames on \n\n, so a spec-compliant SSE
  stream delimited by \r\n\r\n (from a real server) collapsed into one event.
  AppKit's own writer uses \n\n so existing tests were unaffected, but
  expectStream is public API that accepts any Response. Normalize CRLF to LF
  before splitting; add a CRLF regression test.
- Docs: instantiate the plugin CLASS in the attach() snippet (the factory
  returns a descriptor, not an instance), and note that the cache attach()
  seeds is a per-process singleton shared by tests within a file.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Verified and fixed the findings from an independent code review:

- #1 (correctness) expectStream dropped the wire `event:` name when the JSON
  payload carried its own `type` (spread ran after the assignment). Spread the
  payload first, then set `type = name ?? parsed.type`, so a frame like
  `event: error` + `data: {"type":"result"}` reports `error`. Regression test added.
- #2 (contract) `@databricks/appkit/testing` eagerly loads vitest via fixtures
  even for `expectStream`, so vitest is a real requirement. Drop the "optional"
  peerDependenciesMeta and correct the docs sentence.
- #6 (OBO fidelity) the fake `asUser` recorded `asUser: true` unconditionally.
  Enforce the real `Plugin.asUser` token precondition: a request without
  `x-forwarded-access-token` throws `missingToken` (missing user id throws too),
  and the resolved `userId` is recorded on each tool call. Tests now assert both
  directions (well-formed request vs token-less).
- #3 (fidelity) attach() now mirrors AppKit core: registerPlugin plus
  registerToolProvider for real tool providers, without clobbering injected
  fakes. getPlugins()/getPluginNames()/hasPlugin() behave as in production.
- #12 unknown-tool lookup used `tools[name] === undefined`, so a tool named
  "constructor"/"toString" hit Object.prototype. Use Object.hasOwn.
- #5 drop data-less named SSE frames (real clients ignore them).
- #7 re-export the PluginContext type from the testing barrel so
  MockPluginContext.ctx is nameable through the exports map.
- #13 correct the docs: mock.telemetry captures the context's executeTool spans,
  not plugin-level spans (attachContext rebuilds the plugin's own telemetry).
- #4 parseSSEResponse now delegates to the same parseSSEBody as expectStream —
  one parser, no divergence. All 3 analytics.integration call sites still pass.
- #8 reformat template/server/example.test.ts with the template's Prettier so a
  scaffolded app's `npm run format` passes.
- #10 fix the package-doc @example (agentsPlugin._handleStream does not exist).
- #11 add kit tests that exercise attach() end-to-end (cache seed, isReady,
  registration, fake-not-clobbered).

Build passes attw + publint; full appkit suite 3125 passed / 1 pre-existing skip.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

… dep

With vitest declared as a (non-optional) peerDependency, knip recognizes it as
used, so the earlier ignoreDependencies entry is unnecessary. This reverts
knip.json to its original state.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

A required peerDependency has no per-subpath scope: it applied to the whole
@databricks/appkit package, so every production consumer that never imports
the testing kit got an unsatisfied peer (npm 7+ auto-installs vitest into
their tree; pnpm warns) — a wider blast radius than the eager-import bug it
was meant to fix.

Follow appkit's own precedent instead: `vite` backs the ./type-generator
subpath as a normal `dependency`, installed for everyone but loaded only by
importers of that subpath. Do the same for `vitest` and ./testing. vitest is
referenced solely by dist/testing/fixtures.js, never by the main/plugin/core
entry, so a consumer importing createApp never loads it.

Verified end-to-end: scaffolded an app whose own vitest (4.1.9) differs in
major from appkit's dependency (3.2.4), forcing a nested second copy. The
testing kit's vi.fn()/vi.spyOn() mocks and expect(...).toHaveBeenCalled()
assertions work across the two instances (vi spies carry their own call
state), and npm install emits no peer-dep warning. Build passes attw + publint.
Also fold in the template example's Prettier formatting (template uses Prettier,
not Biome) so a scaffolded app's `npm run format` passes.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

The helper builds the REAL PluginContext with faked edges — it does not mock
the context — so the name was misleading. Rename to createTestPluginContext
(and the MockPluginContext type to TestPluginContext), matching the
create*-for-tests convention, and rename the files to test-plugin-context.ts.
Pre-merge and unreleased, so no external consumers are affected.

Also finish the #13 doc-accuracy fix in the shipped JSDoc (not just the docs
page): the telemetry field comment now states it captures the context's spans
(executeTool), not plugin-internal spans — attachContext rebuilds the plugin's
this.telemetry from the real TelemetryManager. These comments ship in
dist/testing/*.d.ts, so IntelliSense previously showed the unqualified claim.

Build passes attw + publint; full appkit suite 3125 passed / 1 pre-existing skip.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Behavior-preserving cleanups in the testing kit:

- createMockRequest reuses createMockWorkspaceClient() instead of an inline
  copy of the same mock client (verified identical).
- createMockServiceContext / createMockUserContext / mockServiceContext inline
  the createMockWorkspaceClient() call into the `||` fallback, so the mock
  client is built only when the caller did not supply one.
- The fake asUser view spreads `...base` and overrides executeAgentTool rather
  than re-declaring getAgentTools.
- expectStream's isSubsequence breaks once the expected sequence is fully
  matched.

No semantic change; typecheck clean and all kit + migrated tests pass.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Exercise @databricks/appkit/testing against real core plugins to validate it
beyond the two agent proof sites and produce usage references:

- analytics.kit.test.ts: cross-plugin executeTool via createTestPluginContext —
  OBO identity (asUser/userId), token-precondition rejection, and per-call
  timeout abort. Needs only the kit (no workspace/ServiceContext).
- genie.kit.test.ts: drives the real _handleSendMessage SSE stream and asserts
  event order with expectStream(...).toEmit(...).

Both add genuinely new coverage (streamed SSE order + OBO dispatch identity were
untested). Full appkit suite 3145 passed / 1 pre-existing skip.

Developer-experience notes (kit wins + friction, e.g. createMockResponse doesn't
compose with expectStream) captured in internal/ for the milestone review.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

The testing kit is entirely plugin-scoped (createTestPluginContext,
attach(plugin), plugin route/tool/SSE assertions), and the page's own
cross-links already pointed into plugins/. Move it next to custom-plugins
and fix the relative links. Keep the heading as 'Testing'; the Plugins
section supplies the context.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Address round-2 review: the kit should be the default way to test a
plugin, not a parallel '*.kit.test.ts' track.

- Fold the three cross-plugin executeTool OBO tests into analytics.test.ts
  and delete analytics.kit.test.ts.
- Upgrade genie.test.ts's SSE test to assert event ORDER via
  expectStream on genie's real event names (message_start, status,
  message_result, query_result), replacing brittle write.mock.calls
  substring checks, and delete genie.kit.test.ts.
- Trim the heavy comment narration from the folded-in tests.
- Re-export createTestPluginContext and expectStream from the test-helpers
  shim.
- Finish the testing-guide move under plugins/ (sidebar position + links).

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

The toEmit swap pinned event order but dropped the payload values the old
substring checks covered (conversationId=new-conv-id, status=ASKING_AI),
which aren't asserted elsewhere. Restore them structurally via collect() +
toMatchObject — keeping the ordering guarantee without brittle substrings.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…equest

createMockRequest returned userWorkspaceClient, serviceWorkspaceClient,
getWarehouseId and getWorkspaceId — fields no production code reads
(plugins resolve those through getWorkspaceClient()/getWarehouseId() from
src/context, which mockServiceContext stands in for). Publishing them via
@databricks/appkit/testing would make four inert fields a permanent public
promise.

The two warehouse cold-start tests (analytics + metric) overrode
mockReq.serviceWorkspaceClient.warehouses.get, which the route never reads
— so they passed on the default RUNNING client without exercising the
warehouse path at all. Route the warehouse client through
mockServiceContext (the real seam) so the tests are live, and drop the
'mock WorkspaceClient' claim from the testing guide.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Every core plugin's actual work runs through getWorkspaceClient(), which the
testing kit did not fake — so a jobs/genie/serving/files plugin crashed on its
first client call and authors hand-rolled nested client literals instead.

createMockWorkspaceClient() fakes the whole facade in three layers:

- The 9 facade members are explicitly typed, so `client.jbos` is a compile
  error. The facade is closed and AppKit-owned, so there is no per-service
  fixture to maintain as the SDK grows.
- Each service is a Proxy minting one memoized vi.fn() per method name, keyed
  by dotted path. `client.jobs.getRun === client.jobs.getRun`, so call
  assertions work, and the legacy view shares the map so one `responses` entry
  covers both — including un-faceted services like `legacy.clusters.list()`.
- `config` and `apiClient` are seeded objects rather than bare Proxies, because
  three of their members must not be mocks: `config.host` is a real string that
  production code builds URLs from and throws on when falsy,
  `apiClient.userAgent()` must be synchronous (a Promise inside a Headers value
  stringifies to "[object Promise]"), and `apiClient.request` resolves {} so
  destructuring its result does not throw.

Two guards keep the Proxy safe. Symbol keys delegate to Reflect.get, and a
passthrough deny-set answers `undefined`. `then` is the load-bearing entry:
without it a service looks thenable, so `await client.jobs` either hangs or
resolves to a mock's return value. ownKeys is left at its default so
util.inspect and toEqual see {} instead of recursing forever.

The three historical canned defaults are byte-identical, because 13 test files
reach them implicitly through mockServiceContext. `currentUser.me` is additive
and load-bearing: ServiceContext.createContext reads `currentUser.id`, so an
unresolved me() is a TypeError and createApp({ client }) cannot boot without it.

getMockFn(client, "jobs.getRun") is the typed assertion path — facade accessors
are legacy-SDK-typed, so expect(client.jobs.getRun).toHaveBeenCalled() does not
typecheck. It mints idempotently, so the handle can be grabbed before the code
under test runs.

The compile-time block is enforced by tsc, not at runtime. It records one
correction to the plan: the SDK types `config.host` as `string | undefined`, so
the contract is that it narrows to a string, not that it is non-optional.

4451 tests pass (+38); the 667 tests reaching the default client indirectly
through mockServiceContext are unchanged.

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

fixtures.ts had its own two-service createMockWorkspaceClient, so the shipped
fixture and the new never-crash builder were near-duplicates. The fixture now
re-exports the builder and the barrel points at its new home.

The blast radius is entirely indirect. Nothing in src imports the exported
fixture by name (connectors/genie/tests/client.test.ts defines its own local
one), but buildServiceContextState calls it as the default client for
mockServiceContext, which 13 test files use. The risk therefore lives in the
default return value, which is why U1 kept the three canned defaults
byte-identical — and why this commit adds the convergence guard that asserts
both halves: jobs/genie now resolve instead of throwing "Cannot read properties
of undefined", while the SQL path those 13 files depend on still succeeds.

createConfigurableMockWorkspaceClient is left byte-for-byte unchanged and only
gains a @deprecated notice. Its bare vi.fn()s return undefined *synchronously*
whereas the new floor returns Promise<undefined>, and its one caller
(analytics.integration.test.ts) can observe that difference; reimplementing it
here would change behaviour for no benefit. It migrates with that suite later.

The jobs suite drops its hand-rolled client literal — the seven method mocks
plus the config.host/authenticate block — onto the builder, which is the proof
the boilerplate actually goes away. Its 57 assertion sites move to a getMockFn
handle because facade accessors are legacy-SDK-typed, so .mockResolvedValue on
them does not typecheck. The factory needs `await vi.hoisted(async ...)` with a
dynamic import, since a hoisted factory runs before the file's imports.

4454 tests pass (+3).

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

LifecycleManager's shutdown sequence was reachable only by killing the process,
so nothing could release AppKit's sockets, timers, pools, cache, and telemetry
and keep running. That is what blocks an app handle's close(), and with it any
test that wants to boot more than once in a file.

The sequence is now a phase runner that returns an exit code, a promise memo,
and two thin callers:

- shutdown() is the signal path, observably unchanged: it arms the same
  unref'd 15s force-exit backstop and still exits 0 on completion, 1 on an
  unexpected throw. The timer stays here deliberately — it is the one thing
  close() must not inherit, since a programmatic caller wants a logged error
  when teardown hangs, not a dead process.
- close() is the programmatic path: it detaches signal handlers, runs the same
  phases under a shorter default budget (5s, not the production 15s), logs the
  phase that was in flight if the budget is spent, and never exits.

Replacing the isShuttingDown boolean with a promise memo is a strict
improvement. The boolean made a second caller return *immediately* while
teardown was still running — harmless for a signal, since the first caller
exits the process anyway, but for close() it would resolve before resources
were released, which is the difference between a correct handle and a
misleading one. The read and the assignment stay in one synchronous statement,
preserving the invariant the boolean was there to protect. One production
behaviour does shift: a second signal now awaits the first teardown.

installSignalHandlers registered anonymous arrows that could never be removed.
The [signal, handler] pairs are now retained and detached individually, never
via removeAllListeners, so a host embedding AppKit keeps its own handlers. The
tests assert that with two managers installed, a.close() leaves b's pair and an
unrelated host listener intact, and that counts return to their pre-install
baseline — which is what stops repeated boots tripping
MaxListenersExceededWarning.

The signal-mid-close race is documented rather than papered over: handlers come
off before the first await, and if a signal still lands it joins the memo and
exits, because it wanted the process dead.

The idempotency test is verified by injection — it fails against the old
return-immediately semantics and passes against the memo. Its first draft did
not: it counted microtask ticks, which cannot distinguish an early return
through close()'s raceWithTimeout wrapper. It now asserts that neither caller
settles until the plugin hook has actually completed.

4463 tests pass (+9); the 14 pre-existing shutdown tests are untouched.

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

createApp acquired sockets, timers, and pools but returned no way to release
them, so the only teardown was killing the process. The LifecycleManager built
at the end of _createApp was constructed and immediately discarded; it is now
retained on the instance and reachable through the handle.

The return type widens from PluginMap<T> to AppHandle<T>, which is PluginMap<T>
plus close() and Symbol.asyncDispose. Widening a return type is
source-compatible for every existing caller, and the cast that produces the
handle already hid instance methods, so close() rides along naturally.
onPluginsReady deliberately keeps PluginMap<T>: it runs before the server
starts, so handing it a close() would invite a footgun for no gain.

The name collision is a real hazard, not a theoretical one. Plugin exports are
installed with Object.defineProperty, and an own property shadows a prototype
method — so a plugin named `close` would silently replace teardown rather than
merely confuse the types. Three layers guard it: Symbol.asyncDispose is
unreachable from a manifest name, so `await using` is always safe;
createAndRegisterPlugin now throws a ConfigurationError naming the offending
plugin; and no plugin in the repo is affected.

Coverage is deliberately unmocked, because the claim is about real resources: a
boot on an ephemeral port serves /health, close() runs the plugin's shutdown
hook, the socket stops accepting, and the SIGTERM listener count returns to its
pre-boot baseline. Also covered: idempotency at the app level, a server-less app
closing cleanly, `await using` releasing at scope exit, and the reserved name
being rejected. Verified by injection — with close() stubbed to a no-op and the
reserved-name guard removed, 5 of the 6 fail.

4469 tests pass (+6).

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

close() released resources but left the singletons pointing at them, so
close() followed by createApp() silently reused what the teardown had just
torn down. This delivers the actual driver — boot, assert, close, repeat.

CacheManager.reset() drops both `instance` and `initPromise`. Clearing only
`instance` is insufficient because getInstance() returns `initPromise` when
`instance` is null, so the next boot would await a promise resolving to the dead
manager. Testing surfaced a third case the plan missed: clearing both is *still*
not enough, because an initialization already in flight runs its continuation
and re-publishes the very instance being discarded. A generation counter now
invalidates that write.

The covering test models PersistentStorage rather than using the default
in-memory storage. This matters: InMemoryStorage.close() only clears a Map and
stays usable, so an in-memory test passes whether or not the reset exists —
which is precisely why the bug hid. Against storage whose close() is terminal,
the way pool.end() is, the test shows the stale manager throwing "Cannot use a
pool after calling end()" and the reset fixing it.

One plan claim is corrected rather than implemented. The plan asserted that
TelemetryManager's never-cleared `shutdownPromise` made a second shutdown()
return a stale promise and skip flushing a re-initialized SDK. It does not:
shutdown() only returns the memo after reassigning it for whatever SDK is
currently live, so a stale resolved promise can be returned only when there is
no SDK to flush. Verified twice — by mocking NodeSDK across three
initialize/shutdown cycles, and by running the original implementation in
isolation. An earlier draft of this commit added a generation counter here too;
it has been reverted, since it fixed nothing and cost a field. What TelemetryManager
did need, and now has, is the static reset() that drops the singleton.

The resets are wired into close() only, never the signal path, where the process
is dying and pointer drops are pure cost. Symmetry is the justification: core
initializes all four in _createApp, so core drops all four. This is a semantic
expansion, not purely a bug fix — a host that closes and then expects
ServiceContext.get() to work will now get an InitializationError.

resetAppKitSingletons() is published from @databricks/appkit/testing for tests
that hand-roll createApp and would otherwise deep-import
../context/service-context to reach ServiceContext.reset(). Both it and
LifecycleManager.close() delegate to one core-side implementation rather than
duplicating the list. resetTestCache() is untouched — it calls clear() on the
existing cache, a different and still-useful operation.

4480 tests pass (+11).

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

One call boots a real AppKit app with no workspace, no credentials, and no
network, and calls it over real HTTP:

  const app = await createTestApp({ plugins: [myPlugin()] });
  const res = await app.post("/api/my-plugin/thing", { body, obo: true });
  await expectStream(res).toEmit("status", "result");
  await app.close();

Four of the setup steps exist only because of hazards found by reading the boot
path, and each has a test that fails without it:

- NODE_ENV is pinned away from "development". Not tidiness: dev mode routes the
  injected `port: 0` through get-port, where portNumbers(0, …) throws a
  RangeError. "development" is refused outright with an explanation rather than
  worked around, since dev mode also boots a real Vite server, downgrades
  resource validation to a warning, and stops filtering dev-only plugins.
- DATABRICKS_WORKSPACE_ID is set, short-circuiting the SCIM probe in
  getWorkspaceId, and internal telemetry is disabled. Both would otherwise fire
  apiClient.request during boot. A canary test asserts zero calls after boot, so
  either regression fails loudly.
- The cache gets explicit in-memory storage. Without it CacheManager builds its
  own workspace client — ignoring the injected one — and probes Lakebase over
  the network, so "no network" would be false.
- The server plugin is reached through a lazy `await import()`, because it runs
  dotenv.config() at module load. A static import would mutate a consumer's
  process.env merely by importing the testing entry point.

process.env is snapshotted wholesale rather than by whitelist, since plugins
read vars the harness cannot enumerate, and restored on close() — including
deleting keys the harness added and restoring a pre-existing DATABRICKS_HOST to
its own value rather than the test default. Teardown also runs from the
boot-failure path, or a plugin whose setup() throws would leak env mutations into
every later test in the file.

Plugin exports live under app.plugins rather than spread onto the handle: `get`
and `delete` are plausible plugin names and would collide with the request
methods.

The request methods return a native Response, so expectStream composes with no
bridge — the dogfooding report's top friction, avoided by construction. `obo`
reuses createMockRequest's OboOption rather than inventing a second convention.

Two corrections to the plan, both found by testing:

- A `strictValidation: false` opt-out was specified and has been dropped as a
  false affordance. enforceValidation computes `shouldThrow = !isDevelopment ||
  strict`, so with NODE_ENV pinned away from "development" validation always
  throws and the flag cannot do anything. The env var is still set as
  belt-and-braces, and a test pins the unconditional behaviour.
- The error-middleware test initially asserted a redacted body. It is not
  redacted: errorHandlerMiddleware hides the message only under
  NODE_ENV=production, and the harness pins "test". Useful for tests — an
  assertion can name the failure — but it means that response is the dev shape,
  which the test now says out loud.

The HTTP suite's probe plugin registers routes through `this.route()`, the way
real plugins do. Registered with raw `router.get()` a rejection escapes
forwardAsyncErrors and hangs the request — correct AppKit behaviour, and worth
having a representative test rather than a misleading one.

4511 tests pass (+31).

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…dogfood both

Audit first: the ./testing subpath was already in both exports maps and the
tsdown entry, vitest is already an optional peer dep, and attw/publint pass. The
real gap was proof that a test needs nothing else, so the integration suites
moved onto the public entry point — that migration *is* the audit. A new
acceptance suite imports only from @databricks/appkit/testing and boots,
requests, asserts a stream, and closes.

Self-referencing the package from inside it needed a tsconfig paths entry.
Resolving the package's own export map made tsc's project root ambiguous
(TS2209), and the alias mirrors how shared and @databricks/lakebase are already
mapped. It also makes source resolution deterministic rather than depending on
the "development" export condition — verified by marker: the subpath resolves to
src, not dist.

createTestPlugin(factory, config) closes the last dogfooding footgun. Reaching
through a descriptor with `new (genie({}).plugin)(config)` skips DEFAULT_CONFIG
and forgets `name`, so the instance under test is configured differently from
the one production builds. It mirrors createAndRegisterPlugin's merge order.
createTestApp does not subsume it: the harness takes descriptors and builds
instances itself, so the unit path needs its own ergonomics.

Dogfooding results, reported as measured rather than as hoped:

- analytics.integration.test.ts: 300 -> 216 lines. Setup/teardown went 104 -> 55,
  against the plan's predicted ~30. Its local getListeningPort helper is gone and
  its 12 mock handles now come from getMockFn. Same 6 tests, same assertions.
- getListeningPort is lifted into the kit, and files/plugin.integration.test.ts
  imports it instead of carrying its own copy.
- server.integration.test.ts moves four of its five blocks to ephemeral ports.
  The fifth keeps its fixed port deliberately, because it asserts the server
  honours a configured one; a comment says so. The removed sleep-100ms waits are
  replaced by getListeningPort, which waits on the listening event instead of
  guessing.

One plan claim corrected: the hardcoded TEST_PORT = 9879 said to collide with
server.integration was already fixed on this branch — analytics had moved to
port: 0. The real fixed ports were the five in server.integration itself, which
is what this commit addresses instead.

Docs lead with createTestApp: a which-harness comparison table, the dotted-path
responses convention, the teardown contract, a "Mocking Databricks services"
section carrying the never-crash floor's honest catch (a misspelled *method*
returns undefined, and a Lakebase pool built on the fake cannot connect), and an
explicit callout that manifest.config.schema is not validated. The
PluginContext/ServiceContext boundary note now says the kit covers the data
plane. The template example gains a createTestApp test.

4519 tests pass (+8). pnpm docs:build is clean.

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

…ings

Verified the kit end to end the way the plan prescribes: pnpm pack:sdk, an app
scaffolded by `databricks apps init` from this repo's template, the tarballs
installed into it, and the suites run with no .env, no credentials, and every
non-loopback socket connection hard-blocked. Nine customer-style tests plus the
template's three pass, and the scaffolded app typechecks against the shipped
.d.ts.

That run corrected a claim this branch had been making. Both the plan's risk
table and the docs said a misspelled *method* slips through the never-crash
floor and only a misspelled *service* is caught. Not so: each facade accessor is
typed against the SDK's own service class, so `client.jobs.getRunz` and
`client.files.anything` are compile errors too. The compile-time block now
asserts that for three services, and the docs say what the real gap is — a
method that exists but has no declared response, or a call that bypasses the
types with a cast.

Also repointed one doc line that told readers to reach the client via
`getWorkspaceClient()`. That is right inside this repo but wrong from the
published entry, where the name currently resolves to Lakebase's unrelated
`getWorkspaceClient(config)`. The docs now use `getExecutionContext().client`,
which is exported and works. The mis-export itself is a main-entry defect,
outside this branch's scope, and is left for a follow-up.

4519 tests pass. Build, docs:build, attw, and publint are clean, and the packed
tarball carries dist/testing/*.js and .d.ts for every new module.

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

Eight reviewers over the branch diff produced 24 findings; 14 were actionable.

The two real defects, both in code this branch added:

- **An orphaned teardown could tear down the *next* app's resources.**
  `close()` races the shutdown phases against a 5s budget, then resets the core
  singletons and resolves. The phases keep running. A plugin `shutdown()` hook
  slower than that budget but inside its own 10s per-plugin budget — which the
  files plugin's drain can be — left phase 5 re-reading the static slots, so it
  either skipped draining this app's cache pool or closed the *following* app's
  storage and shut down its OTEL SDK. The comment claiming the phases had
  "already closed the cache storage" was only true when teardown finished in
  budget. Phase 5 now uses the instances captured before the first await, and a
  test drives the exact 5s-to-10s window (verified by reintroducing the bug).

- **`process.env` restore did not compose across overlapping boots.** Each app
  snapshotted independently, so a second boot captured the first's mutations and
  whichever closed last re-applied them, stranding the harness keys and the first
  app's `env` entries after both apps were gone. Confirmed by probe, in-repo and
  from the packed tarball. There is now one reference-counted baseline: the first
  live app anchors it, the last one to close restores it, and the outcome no
  longer depends on close order.

Also fixed: `server: false` alongside a caller-supplied server plugin is now
refused instead of half-honoured (the plugin still bound a socket while the
handle denied one existed); `AppHandle.close()` declares the `{ timeoutMs }` the
implementation accepts, so the harness no longer casts to reach it; the duplicate
`listeningPort` helper in the close integration suite is gone in favour of the
kit's (two reviewers flagged it); the analytics suite drops an `as never` that
erased `app.plugins` typing; the `clientFns` WeakMap moved above its users; and a
comment claiming "9 typed facade members" over a 7-element array is corrected.

Two of my own tests were weak and are now stronger: the `authenticate` test
wrapped its whole body in `if (mockFn)` and only asserted "was called" — it now
asserts the Authorization header it claims to set — and a close-after-signal test
proved ordering by counting microtask ticks, which cannot see through
`raceWithTimeout`; it uses the same sentinel the sibling test does. A new
compile-time assertion pins that `AppHandle` still satisfies a `PluginMap`
annotation, so a regression in the widening can't pass silently.

Documented rather than changed: a service's methods are callable but not
enumerable, so `'getRun' in client.jobs` is false and `Object.keys` is empty.
Reporting those keys would make `util.inspect` mint a mock per probe, which is
the recursion the default traps exist to avoid. Also documented why
`onPluginsReady` keeps the narrower `PluginMap<T>`.

One finding rejected as a false positive: project-standards reported CLAUDE.md
still documents Biome. It does not — main's own oxlint migration (9538d58)
updated it, and only the pre-merge copy said Biome.

Six findings were demoted to residual risks, chiefly the P1 claim that the mock
resolving `undefined` for undeclared paths lets a test pass while production is
broken. That is the deliberate, documented contract of the never-crash floor, not
a defect; an independent reviewer re-deriving it argues the existing caution
callout is warranted, not that the design changed.

4524 tests pass. Re-verified end to end from a repacked tarball in the
`databricks apps init` app with all non-loopback sockets blocked.

Co-authored-by: Isaac
Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

`createApp()` rejects a plugin whose manifest name is `close`, because plugin
exports are installed as own properties and an own property shadows a prototype
method — so such a plugin would silently replace the app handle's teardown.

The thrown ConfigurationError already names the offending plugin, but nothing
told an author the constraint existed before they hit it. Noted beside where
custom-plugins.md introduces `static manifest`.

Landing this as part of the `feat:` framing for the branch rather than a
BREAKING CHANGE footer: the failure is loud and at boot, not a silent runtime
change, and no plugin in this repo is affected.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

These were added earlier on this branch to work around a locked agent worktree
at .claude/worktrees/, which every tool saw as a second full copy of the repo:
knip reported hundreds of phantom unused exports and failed the pre-commit hook
outright, and a repo-root oxfmt would have rewritten that other branch's tree.

The worktree has since been removed, so the ignores are treating a symptom that
no longer exists and are out of scope for this branch. Verified after removal:
`pnpm knip` and `pnpm check` both exit 0 at the repo root.

.oxfmtrc.json and .oxlintrc.json are now byte-identical to origin/main. The one
remaining knip.json difference — ignoreDependencies: ["vitest"] for
packages/appkit — predates this work and is required because vitest is an
optional peer dependency of the published testing subpath.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

The comments on this branch were far past the repo's own density: reset.ts was
85% comment (35 of 41 lines) for a one-line function, create-test-plugin.ts 63%,
lifecycle-manager.ts 48%, mock-workspace-client.ts 45%. Much of it restated the
code or ran to several paragraphs where a clause would do.

Net 364 comment lines removed. Every file now sits at or below the repo baseline
(main's own sources run 20-42%): mock-workspace-client 45% -> 21%,
create-test-app 36% -> 25%, lifecycle-manager 48% -> 35%, reset.ts 41 -> 13 lines
total, create-test-plugin 63 -> 27.

What was kept is the non-obvious "why" that a maintainer would otherwise delete
and reintroduce a bug: that `then` must stay in the deny-set or `await
client.jobs` hangs; that `ownKeys` stays default or util.inspect mints a mock per
probe; that config.host must be a real string; that the three canned defaults are
byte-identical because 13 suites depend on them; that phase 5 captures its
singletons before the first await; and the four boot hazards behind
createTestApp's setup. Pre-existing comments in files this branch only touched
(fixtures.ts, test-plugin-context.ts, the shutdown() phase list) are left alone —
reverting other people's prose is not this change's business.

Also dropped an unnecessary `as Any` cast in createTestPlugin: DEFAULT_CONFIG is
already declared on PluginConstructor, so the type escape and its explanatory
comment both went.

4524 tests pass; lint, format, and typecheck clean.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>

The mock-workspace-client suite had 37 tests written against the plan's checklist
rather than against behaviours, so eight asserted something a sibling already
covered: getRun memoization twice, config.host being a string twice, the 9-member
facade twice (one a strict subset of the other), a rejecting function response
twice, getMockFn path resolution twice, function-valued responses twice, the
canned defaults twice, and two config-option tests that fit in one.

29 tests now, with no assertion lost — where a dropped test had a unique claim it
was folded into the survivor. Three describe blocks became empty and were
removed; one had only a comment saying its subject could not be tested at
runtime, which the compile-time contract block covers properly.

Note for anyone reading this as a bundle-size fix: it is not one. Tests do not
ship — the packed tarball contains zero test files — and dropping these eight
moved the measured bundle by exactly 0 bytes. The comment trimming in the
previous commit is what actually helped (+8.1% -> +6.9%), because JSDoc is
preserved in the emitted .d.ts.

4516 tests pass.

Signed-off-by: Galymzhan <zhangazy2004@gmail.com>